Field/frame adaptive decoding with field/frame index

ABSTRACT

A moving picture coding method for coding a picture with switching between frame coding and field coding adaptively on a block-by-block basis includes: determining the maximum number of reference indices for field coding for specifying fields which are to be referred to at the time of field coding, using the maximum number of reference indices for frame coding for specifying frames which are to be referred to at the time of frame coding; and assigning to fields the reference indices for field coding for specifying fields which are to be referred to at the time of field coding, within a range of the determined maximum number thereof, using the reference indices for frame coding for specifying frames which are to be referred to at the time of frame coding.

TECHNICAL FIELD

The present invention relates to a moving picture coding method and amoving picture decoding method, and particularly to a coding method anda decoding method for performing inter-picture prediction with referenceto previously coded pictures.

BACKGROUND ART

With development of multimedia applications, it has been popular tohandle integrally all kinds of media information such as video, audioand text. Since digitized images have an enormous amount of data, imageinformation compression techniques are absolutely essential for storageand transmission of such information. It is also important tostandardize such compression techniques for interoperation of compressedimage data. There exist international standards for image compressiontechniques, such as H.261 and H.263 standardized by ITU-T (InternationalTelecommunication Union-Telecommunication Standardization Sector) andMPEG-1, MPEG-2 and MPEG-4 standardized by ISO (InternationalOrganization for Standardization). ITU is now working forstandardization of H.26L as the latest standard for image coding.

Coding of moving pictures, in general, compresses information amount byreducing redundancy in both temporal and spatial directions. Therefore,in inter-picture prediction coding, which aims at reducing the temporalredundancy, motion of a current picture is estimated on a block-by-blockbasis with reference to preceding or subsequent pictures so as togenerate predictive images of the current picture, and then differentialvalues between the obtained predictive images and the current pictureare coded.

Here, the term “picture” represents a single sheet of an image, and itrepresents a frame when used in a context of a progressive image,whereas it represents a frame or a field in a context of an interlacedimage. The interlaced image here is a single frame that is made up oftwo fields having different times respectively. In the process of codingand decoding the interlaced image, a single frame can be handled as aframe, as two fields, or as a frame structure or a field structure onevery block in the frame.

The following description will be given assuming that a picture is aframe in a progressive image, but the same description can be given evenassuming that a picture is a frame or a field in an interlaced image.

FIG. 35 is a diagram for explaining types of pictures and referencerelations between them.

A picture like Picture I1, which is intra-picture prediction codedwithout reference to any pictures, is referred to as an I-picture. Apicture like Picture P10, which is inter-picture prediction coded withreference to one picture, is referred to as a P-picture. And a picture,which can be inter-picture prediction coded with reference to twopictures at the same time, is referred to as a B-picture. B-pictures,like Pictures B6, B12 and B18, can refer to two pictures located inarbitrary temporal directions. Reference pictures can be specified on ablock-by-block basis, on which motion is estimated, and they arediscriminated between a first reference picture which is describedearlier in a bit stream including the coded pictures and a secondreference picture which is described later in the bit stream. However,it is required in order to code and decode above pictures that thereference pictures be already coded and decoded. FIGS. 36A and 36B showexamples of order of pictures in which B-pictures are coded and decoded.FIG. 36A shows a display order of the pictures, and FIG. 36B shows acoding and decoding order reordered from the display order as shown inFIG. 36A. These drawings show that the pictures are reordered so thatthe pictures which are referred to by Pictures B3 and B6 are previouslycoded and decoded.

Next, reference indices for specifying reference pictures will beexplained with reference to FIG. 37 and FIG. 38. For the sake ofsimplicity, numbers for identifying actual pictures are referred to aspicture numbers, while numbers used for specifying reference picturesfor inter-picture prediction are referred to as reference indices.Particularly, indices indicating first reference pictures and secondreference pictures are referred to as first reference indices and secondreference indices, respectively. Default values as shown in FIG. 37 areusually assigned to the reference indices in an initial state, but theassignment can be changed according to commands.

FIG. 37 shows the assignment of two reference indices to the picturenumbers in the initial state of frame coding, and FIG. 38 shows anassignment of reference indices updated using commands from theassignment as shown in FIG. 37. When there is a sequence of picturesordered in coding order, picture numbers are assigned to the picturesstored in a memory in coding order. Commands for assigning the referenceindices to the picture numbers are described in a header of a slice thatis the smaller unit of coding than a picture, and thus the assignmentcan be updated every time one slice is coded. It is possible to use adifferential value between an original picture number and an updatedpicture number as the above command and code an arbitrary number of suchcommands as a command sequence. The first command in the commandsequence is applied to a picture number of a current picture andindicates a picture number corresponding to a reference index number“0”. The second command in the command sequence is applied to thepicture number corresponding to the reference index number “0” andindicates a picture number corresponding to a reference index number“1”. The third command is applied to the picture number corresponding tothe reference index number “1” and indicates a picture numbercorresponding to a reference index number “2”. The same applies to thefourth and the following commands. In the example of the first referenceindices in FIG. 38, a command “−2” is given first and thus the referenceindex number “0” is assigned to the picture with its number “11” byadding “−2” to the picture number “13” of the current picture. Next, acommand “+1” is given and thus the reference index number “1” isassigned to the picture with its number “12” by adding “+1” to thepicture number “11” corresponding to the reference index number “0”. Thefollowing picture numbers are assigned to the reference index numbers inthe same manner. The same goes for the second reference indices.

FIG. 39 is a schematic diagram showing an example of a bit streamgenerated as a result of the above-mentioned coding. As shown in thisfigure, the maximum number of reference indices Max_idx1 for the firstreference pictures (ref1) and the maximum number of reference indicesMax_idx2 for the second reference pictures (ref2) are described in thepicture common information of the bit stream, and the reference indexassignment command sequences idx_cmd1 and idx_cmd2 for ref1 and ref2 aredescribed in the slice header.

A document related to the above conventional technology is ITU-T Rec.H.264|ISO/IEC 14496-10 AVC Joint Final Committee Draft of Joint VideoSpecification (2002 Aug. 10) (P.54, 8.3.6.3 Default index orders/P.56,8.3.6.4 Changing the default index orders).

By the way, as a method of coding an interlaced image, frame coding andfield coding can be used by switching them per block in one picture.This is referred to as Macroblock Adaptive Frame/Field Coding(hereinafter referred to as MBAFF). In this method, frame coding andfield coding can be switched per a pair of two macroblocks placed aboveand below, as shown in FIG. 40. In a case of frame coding, bothmacroblocks are coded as a frame structure, while in a case of fieldcoding, a macroblock consisting of odd-numbered lines and a macroblockconsisting of even-numbered lines are coded separately.

In MBAFF, as shown in FIGS. 41A and 41B, reference pictures are used forreference by switching them between a frame structure and a fieldstructure depending on the coding methods of the macroblock pairs. Whena current macroblock pair is coded as a frame structure as shown in FIG.41A, Pictures P1˜P3 are referred to as frames. When a current macroblockpair is coded as a field structure as shown in FIG. 41B, the picturesare separated into top fields and bottom fields, Pictures P1T˜P3B, andreferred to as respective fields. At this time, the number of referencepictures, which is the number of top and bottom fields, is twice thenumber of frames.

However, the maximum number of reference indices (See max_idx1 andmax_idx2 in FIG. 39) and the command sequences (See idx_cmd1 andidx_cmd2 in FIG. 39) for updating the assignment, which are used forassigning reference indices to respective pictures, cannot be applied toboth frames and fields at the same time. Therefore, there is a problemthat the maximum number of reference indices and the assignment commandscannot be appropriately determined in a case of MBAFF.

SUMMARY OF THE INVENTION

Against this backdrop, the present invention aims at providing a picturecoding method and a picture decoding method for applying referenceindices appropriately to either frame coding or field coding in a caseof MBAFF.

In order to achieve this object, the coding method according to thepresent invention is a moving picture coding method for coding a picturewith switching between frame coding and field coding adaptively on ablock-by-block basis, comprising an assignment step of assigning fieldreference indices to fields using frame reference indices, the fieldreference indices specifying fields which are referred to at the time offield coding, and the frame reference indices specifying frames whichare referred to at the time of frame coding.

According to this structure, frame reference indices can be used forassigning field reference indices. In other words, frame referenceindices can be applied appropriately not only to frame coding but alsoto field coding.

Here, the above-mentioned moving picture coding method may furthercomprise a specification step of specifying two fields that make up eachof the frames specified by each of the frame reference indices, and inthe assignment step, a first value may be assigned to one field having aparity same as a parity of a field including a current block to becoded, out of the specified two fields, as each of the field referenceindices, the first value being obtained by doubling a value of said eachof the frame reference indices, and a second value may be assigned toanother field having a parity different from a parity of the fieldincluding the current block as said each of the field reference indices,the second value being obtained by adding one to said first value.

According to this structure, the value obtained by doubling the value ofthe frame reference index and the value obtained by adding one to thedoubled value are assigned to the field reference indices depending onthe field parity. Therefore, the field reference indices can be assignedextremely easily using the frame reference indices.

Here, the above-mentioned moving picture coding method may furthercomprise a determination step of determining a maximum number of thefield reference indices to be a value obtained by doubling a maximumnumber of the frame reference indices, and in the assignment step, thefield reference indices may be assigned within a range of the determinedmaximum number.

According to this structure, the number obtained by doubling the maximumnumber of frame reference indices can be assigned as the field referenceindices, and thus the effective use of the frame reference indices canbe maximized.

Here, the above-mentioned moving picture coding method may furthercomprise a specification step of specifying two fields that make up eachof the frames specified by each of the frame reference indices, the twofields being a top field and a bottom field, and in the assignment step,a first value may be assigned to the top field, out of the specified twofields, as each of the field reference indices, the first value beingobtained by doubling a value of said each of the frame referenceindices, and a second value may be assigned to the bottom field as saideach of the field reference indices, the second value being obtained byadding one to said first value.

The above-mentioned moving picture coding method may further comprise aspecification step of specifying two fields that make up each of theframes specified by each of the frame reference indices, and in theassignment step, a value same as a value of said each of the framereference indices may be assigned only to one field having a parity sameas a parity of a field including a current block to be coded, out of thespecified two fields, as each of the field reference indices.

Here, the above-mentioned moving picture coding method may furthercomprise an addition step of generating a command sequence indicatinghow to assign the frame reference indices and a command sequenceindicating how to assign the field reference indices independently,coding said two command sequences, and adding said coded commandsequences to a coded signal.

The above-mentioned moving picture coding method, wherein the fieldreference indices consist of top field reference indices and bottomfield reference indices, may further comprise an addition step ofgenerating a command sequence indicating how to assign the framereference indices, a command sequence indicating how to assign the topfield reference indices and a command sequence indicating how to assignthe bottom field reference indices independently, coding said threecommand sequences, and adding said coded command sequences to a codedsignal.

The above-mentioned moving picture coding method may further comprise adetermination step of determining a maximum number of the fieldreference indices, and in the assignment step, the field referenceindices may be assigned to fields within a range of the determinedmaximum number using the frame reference indices.

Here, in the determination step, the maximum number of the fieldreference indices may be determined to be a value obtained by doubling amaximum number of the frame reference indices.

According to this structure, the frame reference indices can be usedeffectively at the maximum for the field reference indices within thenumber obtained by doubling the maximum number of frame referenceindices.

Here, in the determination step, the maximum number of the fieldreference indices may be determined to be a value same as a maximumnumber of the frame reference indices.

According to this structure, the frame reference indices can be usedeffectively at the maximum for the field reference indices within thenumber same as the maximum number of frame reference indices.

Here, the above-mentioned moving picture coding method may furthercomprise an addition step of determining a maximum number of the framereference indices independently of the maximum number of the fieldreference indices, coding said two maximum numbers, and adding saidcoded maximum numbers to a coded signal.

According to this structure, the maximum number of the field referenceindices can be determined independently of the maximum number of theframe reference indices, and the decoding apparatus can notify thedetermined maximum number via a coded signal.

Here, the above-mentioned moving picture coding method, wherein thefield reference indices consist of top field reference indices andbottom field reference indices, may further comprise an addition step ofdetermining a maximum number of the frame reference indices, a maximumnumber of the top field reference indices and a maximum number of thebottom field reference indices independently, coding said three maximumnumbers, and adding said coded maximum numbers to a coded signal.

As described above, according to the coding method of the presentinvention, the reference indices, the maximum number of the referenceindices and the commands that are originally intended for frame codingcan also be utilized appropriately in field coding, in a case of MBAFF.

Also, the moving picture decoding method, the moving picture codingapparatus, the moving picture decoding apparatus and the program of thepresent invention have the same structures, functions and effects asmentioned above.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram showing a structure of a coding apparatus in afirst embodiment of the present invention.

FIG. 2 is an illustration showing an example of correspondences betweenpicture numbers and first and second reference indices in a case offrame coding of macroblocks (MB).

FIG. 3 is an illustration showing an example of correspondences betweenthe first and second reference indices, commands and picture numbers.

FIG. 4 is an illustration showing an example of assigning the first andsecond reference indices to the picture numbers of fields in a case offield coding of macroblocks.

FIG. 5 is a flowchart showing the processing of assigning referenceindices and commands executed by a reference index/picture numberconversion unit in the coding apparatus.

FIG. 6 is a flowchart showing the processing of assigning referenceindices for field coding to fields.

FIG. 7 is a block diagram showing a structure of a decoding apparatus inthe first embodiment of the present invention.

FIG. 8 is a block diagram showing a structure of a coding apparatus in asecond embodiment of the present invention.

FIG. 9 is an illustration showing an example of assigning the first andsecond reference indices to picture numbers of fields in a case of fieldcoding of a macroblock.

FIG. 10 is a flowchart showing the processing of assigning referenceindices executed by a reference index/picture number conversion unit inthe coding apparatus.

FIG. 11 is a block diagram showing a structure of a decoding apparatusin the second embodiment of the present invention.

FIG. 12 is a block diagram showing a structure of a coding apparatus ina third embodiment of the present invention.

FIG. 13 is an illustration showing an example of assigning the first andsecond reference indices to picture numbers of fields in a case of fieldcoding of a macroblock.

FIG. 14 is a block diagram showing a structure of a decoding apparatusin the third embodiment of the present invention.

FIG. 15 is a block diagram showing a structure of a coding apparatus ina fourth embodiment of the present invention.

FIG. 16 is an illustration showing an example of assigning the first andsecond reference indices to picture numbers of fields in a case of fieldcoding of a macroblock.

FIG. 17 is a block diagram showing a structure of a coding apparatus ina fifth embodiment of the present invention.

FIG. 18 is an illustration showing an example of assigning the first andsecond reference indices to picture numbers of fields in a case of fieldcoding of a macroblock.

FIG. 19 is a flowchart showing the processing of assigning referenceindices executed by a reference index/picture number conversion unit inthe coding apparatus.

FIG. 20 is a block diagram showing a structure of a decoding apparatusin the fifth embodiment of the present invention.

FIG. 21 is a diagram showing a data structure of a bit stream in a sixthembodiment of the present invention.

FIG. 22 is an illustration showing an example of assigning the first andsecond reference indices to picture numbers of fields in a case of fieldcoding of a macroblock.

FIG. 23 is a block diagram showing a structure of a coding apparatus ina seventh embodiment of the present invention.

FIG. 24 is a diagram showing an example of a data structure of a bitstream.

FIG. 25 is an illustration showing an example of assigning the first andsecond indices to picture numbers of fields in a case of field coding ofa macroblock.

FIG. 26 is a diagram showing an example of correspondences betweenreference indices, commands and picture numbers of fields specificallyapplied to top fields and bottom fields respectively in a case of fieldcoding.

FIG. 27 is a flowchart showing the processing of assigning referenceindices and commands in a case of a mixture of frame coding and fieldcoding.

FIG. 28 is a block diagram showing a structure of a decoding apparatusin the seventh embodiment of the present invention.

FIG. 29 is a diagram showing another example of a data structure of abit stream.

FIG. 30A is an illustration of a physical format of a recording mediumfor storing a program for realizing the moving picture coding method andmoving picture decoding method in each of the embodiments by a computersystem.

FIG. 30B is an illustration of a front view and a cross-section view ofa recording medium for storing a program for realizing the movingpicture coding method and moving picture decoding method in each of theembodiments by a computer system.

FIG. 30C is an illustration of a computer system for use with arecording medium for storing a program for realizing the moving picturecoding method and moving picture decoding method in each of theembodiments.

FIG. 31 is a block diagram showing an overall configuration of a contentsupply system.

FIG. 32 is an external view of a mobile phone.

FIG. 33 is a block diagram showing a structure of the mobile phone.

FIG. 34 is a diagram showing an example of a digital broadcastingsystem.

FIG. 35 is a schematic diagram for explaining reference relationsbetween pictures in a background art.

FIG. 36A is a schematic diagram for explaining reordering of pictures inthe background art.

FIG. 36B is a schematic diagram for explaining reordering of pictures inthe background art.

FIG. 37 is a schematic diagram for explaining how to assign picturenumbers to reference indices in the background art.

FIG. 38 is a schematic diagram showing assignment of reference indicesupdated from the assignment as shown in FIG. 37 using commands in thebackground art.

FIG. 39 is a schematic diagram for explaining a structure of a bitstream in the background art.

FIG. 40 is an illustration of macroblock pairs in cases of frame codingand field coding.

FIG. 41A is an illustration showing reference frames in frame coding andreference fields in field coding.

FIG. 41B is an illustration showing reference frames in frame coding andreference fields in field coding.

DETAILED DESCRIPTION OF THE INVENTION First Embodiment

Overview of Coding Apparatus and Decoding Apparatus

First, an overview of a coding apparatus and a decoding apparatus in thepresent embodiment will be given.

When performing macroblock adaptive frame/field coding (MBAFF), thecoding apparatus and the decoding apparatus in the present embodimenthandle the maximum number of reference indices and a command sequence inthe following manners (1.1) and (1.2), respectively. Here, the referenceindices and the commands are same as those as shown in FIG. 38, and themaximum number of the reference indices are same as those as shown inFIG. 39.

(1.1) As for the maximum number of the reference indices, the codingapparatus describes the maximum number of reference indices for framecoding (frame reference indices) in a bit stream to be transmitted whenfield coding and frame coding are mixed. The coding apparatus handlesthe maximum number of reference indices as the number of availablereference indices in frame coding, while, in field coding, it considersthe value obtained by doubling the maximum number for frame coding asthe number of reference indices for field coding (field referenceindices). For example, when the reference indices for frame coding 0˜2are assigned, the maximum number of reference indices is “3”. In a caseof frame coding, this number indicates the actual maximum number itself.In a case of field coding, the number “6” obtained by doubling themaximum number of reference indices for frame coding “3” is consideredas the maximum number of reference indices for field coding. The sameapplies to the decoding apparatus.

(1.2) As for the command sequence, the coding apparatus describescommands for frame coding in a bit stream to be transmitted. The codingapparatus assigns the reference indices for frame coding in a case offrame coding, as explained using FIG. 38. Note that if the commandsequence is not coded, correspondences between picture numbers andreference indices are established in the manner of default assignment asshown in FIG. 37.

In a case of field coding, the assignment of reference indices isupdated for field coding based on the reference indices for frame codingwhich are already assigned.

To be more specific, the value obtained by doubling the value of thereference index for frame coding is assigned to a field of the sameparity as a field including a current macroblock to be coded, among twofields that make up one frame, while the value obtained by doubling thevalue of the reference index for frame coding and adding 1 (x2+1) isassigned to another field of the opposite parity, as a reference indexfor field coding, respectively (See FIG. 4). Here, “parity” means an oddor even quality of a field (distinction between a top field consistingof odd-numbered lines and a bottom field consisting of even-numberedlines).

In other words, when a current macroblock to be coded belongs to a topfield, the value obtained by doubling a value of a reference index forframe coding is assigned to a top field among two fields, while thevalue obtained by adding 1 to the doubled value (x2+1) is assigned to abottom field among the two fields. When a current macroblock belongs toa bottom field, the value obtained by doubling the value of thereference index for frame coding is assigned to a bottom field among twofields, while the value obtained by adding 1 to the doubled value (x2+1)is assigned to a top field among the two fields.

On the other hand, the decoding apparatus decodes the maximum number ofreference indices for frame coding and the assignment commands includedin the transmitted bit stream, and assigns the reference indices to thereference pictures, using the maximum number and the commands, inexactly the same manner as the coding apparatus.

Structure of Coding Apparatus

Next, the structure of the coding apparatus will be explained.

FIG. 1 is a block diagram showing the structure of the moving picturecoding apparatus in the first embodiment of the present invention. Usingthe figure, (1) an overview of coding and (2) an assignment method ofreference indices and commands for frame coding and an assignment methodof reference indices for field coding will be explained in this order.

(1) Overview of Coding

It is assumed here that a current picture represents either a frame or afield to be coded, and thus the overview of coding which is common toboth frame coding and field coding will be explained below.

A moving picture to be coded is inputted to a picture memory 101 on apicture-by-picture basis in display order, and the inputted pictures arereordered in coding order. FIGS. 36A and 36B are diagrams showing anexample of reordering of pictures. FIG. 36A shows an example of picturesin display order, and FIG. 36B shows an example of the picturesreordered in coding order. Here, since Pictures B3 and B6 refer bothtemporally preceding and subsequent pictures, the reference picturesneed to be coded before coding these current pictures and thus thepictures are reordered in FIG. 36B so that Pictures P4 and P7 are codedearlier. Each of the pictures is divided into blocks called macroblocksof horizontal 16×vertical 16 pixels, for example, and the followingproceeding is performed on a block-by-block basis.

An input image signal read out from the picture memory 101 is inputtedto a difference calculation unit 112, a difference between the inputimage signal and the predicted image signal that is an output from amotion compensation coding unit 107 is calculated, and the obtaineddifference image signal (residual error signal) is outputted to aprediction error coding unit 102. The prediction error coding unit 102performs image coding processing such as frequency transformation andquantization, and outputs a coded residual error signal. The codedresidual error signal is inputted to a prediction error decoding unit104, which performs image decoding processing such asinverse-quantization and inverse-frequency transformation and outputs adecoded residual error signal. An addition unit 111 adds the decodedresidual error signal and the predicted image signal to generate areconstructed image signal, and stores, in a picture memory 105, thereconstructed signals which could be referred in the followinginter-picture prediction out of the obtained reconstructed imagesignals.

On the other hand, the input image signal read out per macroblock fromthe picture memory 101 is also inputted into a motion vector estimationunit 106. Here, the reconstructed image signals stored in the picturememory 105 are searched to estimate an image area which is the closestto the input image signal and determine a motion vector pointing to theposition of the image area. The motion vector estimation is performedper block that is a part of a macroblock, and the obtained motionvectors are stored in a motion vector storage unit 108. At this time,since a plurality of pictures can be used for reference in H.26L whichis now under consideration for standardization, identification numbersfor specifying reference pictures are required per block. Theidentification numbers are referred to as reference indices, and areference index/picture number conversion unit 109 establishescorrespondences between the reference indices and the picture numbers ofthe pictures stored in the picture memory so as to allow specificationof the reference pictures.

The motion compensation coding unit 107 extracts the image area that ismost suitable for the predicted image from among the reconstructed imagesignals stored in the picture memory 105, using the motion vectorsestimated by the above-mentioned processing and the reference indices.It is judged at this time which is more efficient, frame predictivecoding or field predictive coding, in each macroblock, and then codingis performed using the selected method. The bit stream generation unit103 performs variable length coding for the coded information such asthe reference indices, the motion vectors and the coded residual errorsignals outputted as a result of the above series of processing so as toobtain a bit stream to be outputted from this coding apparatus.

The flow of operations in a case of inter-picture prediction coding hasbeen described above, but a switch 112 and a switch 113 switch betweeninter-picture prediction coding and intra-picture prediction coding. Ina case of intra-picture prediction coding, a predicted image is notgenerated by motion compensation, but a difference image signal isgenerated by calculating a difference from a predicted image in acurrent area which is generated from a coded area in the currentpicture. The prediction error coding unit 102 converts the differenceimage signal into the coded residual error signal in the same manner asinter-picture prediction coding, the bit stream generation unit 103performs variable length coding for the signal to obtain a bit stream tobe outputted.

(2) Assignment Method of Reference Indices

Example of Assignment of Reference Indices

First, FIG. 2˜FIG. 4 show examples of assignment methods of referenceindices for frame coding and reference indices for field coding.

FIG. 2 shows an example of assignment of default reference indices in acase where frame coding is performed on a block in a current picture tobe coded, and the reference indices are assigned to the picture numbersin decreasing order of the picture number. The reference indices arealways assigned in this manner when assignment commands are not coded.FIG. 3 shows an example where the default reference indices as shown inFIG. 2 are updated using the assignment commands. Since “−2” is givenfirst as a command, a picture with its picture number “11” is assignedto the reference index number “0” by adding “−2” to the current picturenumber “13”. Next, “+1” is given as a command, a picture with itspicture number “12” is assigned to the reference index number “1”. Eachof the following picture numbers is assigned in the same manner. Thesame applies to the second reference indices. The following will beexplained based on FIG. 2 showing the default assignment, but thereference indices can be assigned in exactly the same manner even if thedefault assignment is updated by commands. Note that the above commandsare just an example, and the reference indices can be assigned inexactly the same manner even if the default assignment is updated bycommands for other assignments than the above example.

FIG. 4 is an illustration showing correspondences of the first andsecond reference indices for top field coding (top field referenceindices) and bottom field coding (bottom field reference indices),respectively, updated from the first and second reference indices forframe coding as shown in FIG. 2, according to the above (1.1) and (1.2).FIG. 4 shows that the values obtained by doubling those of the referenceindices for frame coding are assigned to the fields of the same parityas the field including a current macroblock, while the values obtainedby doubling those of the reference indices for frame coding and adding 1(x2+1) are assigned to the fields of the opposite parity.

In the present embodiment, if field coding and frame coding are mixed inone picture, the maximum number of reference indices for field coding ishandled as the value obtained by doubling that for frame coding, andthus the number of indices in FIG. 4 is “6”, whereas the number ofindices in FIG. 2 is “3”.

Processing of Assigning Reference Indices

FIG. 5 is a flowchart showing the processing of assigning referenceindices executed by the reference index/picture number conversion unitof the coding apparatus.

The reference index/picture number conversion unit 109 performs theprocessing of assigning reference indices per slice in a case of MBAFF.Here, a slice means each of one or more areas which make up a picture.The reference index/picture number conversion unit 109 omits all theprocessing in this figure when there is no change of reference indices(in a case of default).

As shown in this figure, the reference index/picture number conversionunit 109 first performs the processing of assigning reference indicesand commands for frame coding to frames (S11). Since this processing issame as that as described using FIG. 37, it is omitted here. Next, thereference index/picture number conversion unit 109 judges whether or notframe coding and field coding are mixed in the slice (S12), and if theyare mixed, it performs the processing of assigning reference indices forfield coding (S13).

FIG. 6 is a flowchart showing the processing of assigning referenceindices to fields based on the correspondences between reference indicesfor frame coding and reference indices for field coding. In this figure,a variable j is 1 and 2 (j=1, 2) for B-pictures and j is 1 (j=1) forP-pictures, and max_idxj indicates the maximum number of the jthreference indices for frame coding, and idxj(i) indicates the value ofthe ith-jth reference index for frame coding, respectively. Loop 2 canbe applied commonly to B-pictures and P-pictures. Loop 1 has iterationsfor the maximum number of reference indices for frame coding (max_idxj),and two reference indices for field coding are assigned for everyiteration of loop 1.

The processing of assigning two reference indices for field coding usingone-iteration of loop 1, that is, one reference index for frame coding,will be explained below. The reference index/picture number conversionunit 109 reads out the value of the ith-jth reference index for framecoding idxj(i) assigned in S11 of FIG. 5 (S23), and judges whether thecurrent macroblock belongs to the top field or not (S26).

When the current macroblock is judged to belong to the top field, thevalue obtained by doubling that of the reference index for frame codingidxj(i) (S27) is assigned to the top field out of the two fieldsspecified in S25 (S28), and the value obtained by doubling the valueidxj(i) and adding 1 (S29) is assigned to the bottom field out of thetwo fields specified in S25 (S30).

When the current macroblock is judged to belong to the bottom field, thevalue obtained by doubling that of the reference index for frame codingidxj(i) (S31) is assigned to the bottom field out of the two fieldsspecified in S25 (S32), and the value obtained by doubling the valueidxj(i) and adding 1 (S33) is assigned to the top field out of the twofields specified in S25 (S34).

As described above, the value obtained by doubling the value of thereference index for frame coding and the value obtained by adding 1 tothe doubled value (x2+1) are assigned to the reference indices for fieldcoding. Therefore, as shown in FIG. 4, the value obtained by doublingthe maximum number of reference indices for frame coding (max_idxj) isassigned to the maximum number of reference indices for field coding.

In coding a macroblock, reference indices for field coding used asreference fields in the field-coded macroblock are set in a bit streamas ref1 and ref2 (See FIG. 39). On the other hand, reference indices forframe coding used as reference frames in the frame-coded macroblock areset in a bit stream as ref1 and ref2 (See FIG. 39).

The number of reference indices for frame coding is 3 in the example ofFIG. 2, whereas the number of reference indices for field coding is 6 inthe example of FIG. 4.

FIG. 6 shows the processing of assigning reference indices for fieldcoding to each current picture to be field-coded, but a table may beprepared in advance. To be more specific, the present embodiment may bestructured so as to create a table indicating correspondences betweenreference indices for frame coding and picture numbers of framesaccording to commands, and further, by assigning the reference indicesfor top field coding and bottom-field coding respectively in the samemanner as shown in FIG. 6, to create a table indicating correspondencesbetween reference indices for top field coding and picture numbers offields and a table indicating correspondences between reference indicesfor bottom field coding and picture numbers of fields. Once these tablesare created at the beginning of coding or decoding pictures, thereference pictures can be determined only with reference to thereference indices indicated in these tables.

Structure of Decoding Apparatus

FIG. 7 is a block diagram showing a structure of a decoding apparatus inthe first embodiment of the present invention. Using this figure, (1) anoverview of decoding and (2) processing of converting reference indiceswill be explained in this order. Here, it is assumed that a bit streamis transmitted from the coding apparatus as shown in FIG. 1 to thepresent decoding apparatus.

(1) Overview of Decoding

First, a bit stream analysis unit 201 extracts various information fromthe inputted bit stream: the maximum number of reference indices from apicture common information area, command sequences for reference indexassignment from a slice header area, and reference indices, motionvector information and a coded residual error signal from a coded blockinformation area, respectively.

The maximum number of reference indices and the command sequences forreference index assignment extracted by the bit stream analysis unit 201are outputted to a reference index/picture number conversion unit 206,the reference indices are outputted to a motion compensation decodingunit 204, the motion vector information is outputted to a motion vectorstorage unit 205, and the coded residual error signal is outputted to aprediction error decoding unit 202, respectively.

The prediction error decoding unit 202 performs image decodingprocessing such as inverse-quantization and inverse-frequencytransformation for the inputted coded residual error signal, and outputsa decoded residual error signal. The addition unit 207 adds the decodedresidual error signal and the predicted image signal outputted from themotion compensation decoding unit 204 to generate a reconstructed imagesignal. The obtained reconstructed image signal is stored in a picturememory 203 for use for reference in the following inter-pictureprediction and output for display.

The motion compensation decoding unit 204 extracts an image area whichis most suitable as a predicted image from the reconstructed imagesignals stored in the picture memory 203, using the motion vectorsinputted from the motion vector storage unit 205 and the referenceindices inputted from the bit stream analysis unit 201. At this time,the reference index/picture number conversion unit 206 specifies thereference pictures in the picture memory 203 based on thecorrespondences between the given reference indices and the picturenumbers. If field coding is mixed, it specifies reference fields afterconverting the reference indices for frame coding into the referenceindices for field coding.

Further, the motion compensation decoding unit 204 performs pixel valueconversion processing such as interpolation processing by linearprediction on pixel values in the extracted image area so as to generatethe ultimate predicted image. The decoded image generated through theabove-mentioned series of processing is stored in the picture memory 203and outputted as a picture signal for display according to displaytiming.

The flow of operations in a case of inter-picture prediction decodinghas been described above, but a switch 208 switches betweeninter-picture prediction decoding and intra-picture prediction decoding.In a case of intra-picture decoding, a predicted image is not generatedby motion compensation, but a decoded image is generated by generating apredicted image of a current area to be decoded from a decoded area inthe same picture and adding the predicted image. The decoded image isstored in the picture memory 203, as is the case with the inter-pictureprediction decoding, and outputted as a picture signal for displayaccording to display timing.

(2) Processing of Converting Reference Indices

The reference index/picture number conversion unit 206 assigns picturenumbers and reference indices using the inputted maximum number ofreference indices and commands for reference index assignment. They areassigned in exactly the same manner as the coding apparatus. In thepresent embodiment, the value obtained by doubling the maximum number ofreference indices for frame coding is used as the maximum number ofreference indices for field coding. Therefore, the assignment for framecoding as shown in FIG. 2 turns to be the assignment as shown in FIG. 4for field coding.

As described above, according to the coding apparatus and the decodingapparatus in the present embodiment, the maximum number of referenceindices and the assignment commands for frame coding, if only they arecoded in a bit stream, can be applied appropriately not only to framecoding but also to field coding in a case of MBAFF. Also, the valueobtained by doubling the maximum number of reference indices for framecoding is used as the maximum number for field coding, all the fieldsstored in the memory can be used effectively for coding and decoding.

Second Embodiment

Overview of Coding Apparatus and Decoding Apparatus

First, an overview of a coding apparatus and a decoding apparatus in thepresent embodiment will be explained.

The coding apparatus and the decoding apparatus in the presentembodiment perform MBAFF, and for that purpose, they handle the maximumnumber of reference indices and a command sequence in the followingmanners (2.1) and (2.2), respectively.

(2.1) Since the maximum number of reference indices is same as (1.1) asdescribed at the outset of the first embodiment, the explanation thereofis omitted.

(2.2) As for the command sequence, the coding apparatus describescommands for frame coding in a bit stream to be transmitted. Asdescribed using FIG. 37 and FIG. 38, the coding apparatus assignsreference indices for frame coding for the purpose of frame coding. Notethat correspondences of the reference indices are established in themanner of the default assignment, as described using FIG. 37, if thecommand sequence is not coded.

Further, for the purpose of field coding, the assignment of referenceindices is updated based on the assigned reference indices for framecoding.

In the present embodiment, differently from the first embodiment,regardless of whether a current macroblock to be coded is in a top fieldor a bottom field, the value obtained by doubling the value of referenceindex for frame coding is assigned to a top field out of two fields thatmake up one frame, while the value obtained by doubling the referenceindex for frame coding and adding 1 (x2+1) is assigned to a bottomfield, respectively, as reference indices for field coding (See FIG. 9).

Structure of Coding Apparatus

FIG. 8 is a block diagram showing the structure of the coding apparatusin the second embodiment of the present invention. The coding apparatusin this figure is different from that in FIG. 1 in that the formerincludes a reference index/picture number conversion unit 109 a, insteadof the reference index/picture number conversion unit 109. The samepoints as those in FIG. 1 are omitted, and the following explanationwill focus on the different points. The reference index/picture numberconversion unit 109 a is different from FIG. 1 only in that the formerestablishes a mapping (assignment of reference indices) ofabove-mentioned (2.2), not a mapping of (1.2).

Example of Assignment of Reference Indices

FIG. 9 is an illustration showing correspondences of the first andsecond reference indices for field coding, updated from the first andsecond reference indices for frame coding as shown in FIG. 2, accordingto the above (2.1) and (2.2). As shown in FIG. 9, the mapping executedby the reference index/picture number conversion unit 109 a in thepresent embodiment is not separate assignment of reference indices fortop field coding and bottom field coding, but common assignment for bothtop field coding and bottom field coding.

In the present embodiment, when field coding and frame coding are mixedin one picture, the value obtained by doubling the maximum number ofreference indices for frame coding is handled as the value for fieldcoding, and thus the number of indices in FIG. 2 is “3”, whereas thenumber of indices in FIG. 9 is “6”.

Processing of Assigning Reference Indices

FIG. 10 is a flowchart showing the processing of assigning referenceindices executed by the reference index/picture number conversion unitin the coding apparatus.

In FIG. 10, the same step numbers are assigned to the same processing asthat in FIG. 6, and the flowchart in FIG. 10 is different from that inFIG. 6 in that S26 and S31˜S34 in FIG. 6 are deleted and S27 is executednext to S23 in FIG. 10. Due to these differences, the number ofreference indices obtained by doubling the number of the referenceindices for frame coding is assigned as the reference indices for fieldcoding, and further the reference indices for field coding are assignedcommonly for both top field coding and bottom field coding, as shown inFIG. 9.

Structure of Decoding Apparatus

FIG. 11 is a block diagram showing the structure of the decodingapparatus in the second embodiment of the present invention. Thedecoding apparatus in FIG. 11 is different from that in FIG. 7 in thatthe former includes a reference index/picture number conversion unit 206a, instead of the reference index/picture number conversion unit 206.The reference index/picture number conversion unit 206 a is differentfrom FIG. 7 only in that the former converts the reference indicesaccording to the mapping of (2.2), not the mapping of (1.2).

Processing of Converting Reference Indices

The reference index/picture number conversion unit 206 a assigns picturenumbers and reference indices using the inputted maximum number ofreference indices and the reference index assignment commands. They areassigned in exactly the same manner as the coding apparatus. In thepresent embodiment, the value obtained by doubling the value of themaximum number of reference indices for frame coding is used as themaximum number of reference indices for field coding. Therefore, theassignment for frame coding as shown in FIG. 2 turns to be theassignment for field coding as shown in FIG. 9.

Third Embodiment

Overview of Coding Apparatus and Decoding Apparatus

First, the overview of the coding apparatus and the decoding apparatusin the present embodiment will be explained.

The coding apparatus and the decoding apparatus in the presentembodiment perform MBAFF, and for that purpose, they handle the maximumnumber of the reference indices and the command sequence in thefollowing manners (3.1) and (3.2).

(3.1) As for the maximum number of the reference indices, the codingapparatus describes the maximum number of reference indices for framecoding in a bit stream to be transmitted when field coding and framecoding are mixed. The coding apparatus handles this maximum number ofreference indices as the number of available reference indices in framecoding, and, in field coding, it also handles the number for framecoding as the number of reference indices for field coding. For example,if the maximum number of reference indices for frame coding is 3, thecoding apparatus also handles the maximum number of reference indicesfor field coding as 3.

(3.2) As for the command sequence, since it is handled in the samemanner as (1.2) as described at the outset of the first embodiment, theexplanation thereof is omitted. However, the same value is used as themaximum number of reference indices given by (3.1) for both frame codingand field coding, so only the same number of reference indices as thatas shown in FIG. 2 can be applied to field coding (See FIG. 13).

Structure of Coding Apparatus

FIG. 12 is a block diagram showing the structure of the coding apparatusin the third embodiment of the present invention. The coding apparatusin this figure is different from that in FIG. 1 in that the formerincludes a reference index/picture number conversion unit 109 b, insteadof the reference index/picture number conversion unit 109. The referenceindex/picture number conversion unit 109 b is different from that inFIG. 1 only in that the former handles the number of reference indicesaccording to (3.1), not to (1.1).

Example of Reference Index Assignment

FIG. 13 is an illustration showing correspondences of the first andsecond reference indices for field coding, updated from the first andsecond reference indices for frame coding as shown in FIG. 2, accordingto the above (3.1) and (3.2). As shown in FIG. 13, the mapping executedby the reference index/picture number conversion unit 109 b in thepresent embodiment is separate assignment of reference indices to topfields and bottom fields in the same manner as the first embodiment, butis different in that the maximum number of reference indices for fieldcoding is same as the maximum number of reference indices for framecoding.

Structure of Decoding Apparatus

FIG. 14 is a block diagram showing the structure of the decodingapparatus in the third embodiment of the present invention. The decodingapparatus in FIG. 14 is different from that in FIG. 7 in that the formerincludes a reference index/picture number conversion unit 206 b, insteadof the reference index/picture number conversion unit 206. The referenceindex/picture number conversion unit 206 b is different from FIG. 7 onlyin that the former performs the reference index conversion processingaccording to the maximum number described in (3.2), not the maximumnumber described in (1.1).

Fourth Embodiment

Overview of Coding Apparatus and Decoding Apparatus

First, an overview of the coding apparatus and the decoding apparatus inthe present embodiment will be explained.

The coding apparatus and the decoding apparatus in the presentembodiment perform MBAFF, and for that purpose, they handle the maximumnumber of reference indices and the command sequence in the followingmanners (4.1) and (4.2).

(4.1) As for the maximum number of reference indices, since it ishandled in the same manner as (3.1) as described at the outset of thethird embodiment, the explanation thereof is omitted.

(4.2) Since it is same as (2.2) as described at the outset of the secondembodiment, the explanation thereof is omitted. However, the same valueis used as the maximum number of reference indices given by (4.1) forboth frame coding and field coding, so only the same number of referenceindices as that as shown in FIG. 2 can be applied for field coding (SeeFIG. 16).

Structure of Coding Apparatus

FIG. 15 is a block diagram showing the structure of the coding apparatusin the fourth embodiment of the present invention. The coding apparatusin this figure is different from that in FIG. 8 in that the formerincludes a reference index/picture number conversion unit 109 c, insteadof the reference index/picture number conversion unit 109 a. Thereference index/picture number conversion unit 109 c is different fromthat in FIG. 8 only in that the former handles the maximum number ofreference indices according to (4.1), not to (2.1).

Example of Reference Index Assignment

FIG. 16 is an illustration showing correspondences of the first andsecond reference indices for field coding, updated from the first andsecond reference indices for frame coding as shown in FIG. 2, accordingto the above (4.1) and (4.2). As shown in FIG. 16, the mapping executedby the reference index/picture number conversion unit 109 c in thepresent embodiment is assignment of common reference indices for bothtop field coding and bottom field coding, in the same manner as thesecond embodiment, but is different in that the maximum number ofreference indices for field coding is same as the maximum number ofreference indices for frame coding.

Structure of Decoding Apparatus

The decoding apparatus in the present embodiment may be same as thedecoding apparatus in the second embodiment. However, the former isdifferent from the latter in that the former handles the maximum numberof reference indices for field coding as the same number as the maximumnumber of reference indices for frame coding, not the doubled number.

Fifth Embodiment

Overview of Coding Apparatus and Decoding Apparatus

First, the overview of the coding apparatus and the decoding apparatusin the present embodiment will be explained.

The coding apparatus and the decoding apparatus in the presentembodiment perform MBAFF, and for that purpose, they handle the maximumnumber of the reference indices and the command sequence in thefollowing manners (5.1) and (5.2).

(5.1) As for the maximum number of reference indices, since it ishandled in the same manner as (3.1) as described at the outset of thethird embodiment, the explanation thereof is omitted.

(5.2) As for the command sequence, the coding apparatus describescommands for frame coding in a bit stream to be transmitted. Asdescribed using FIG. 37 and FIG. 38, the coding apparatus assignsreference indices for frame coding for the purpose of frame coding. Notethat correspondences of the reference indices are established by thedefault assignment method, as described using FIG. 37, if the commandsequence is not coded.

Further, for the purpose of field coding, the assignment of referenceindices is updated based on the assigned reference indices for framecoding.

In the present embodiment, differently from the first embodiment, thevalue of reference index for frame coding is assigned to a field of thesame parity as that of a current macroblock to be coded, out of twofields that make up one frame, as a reference index for field coding,while no value is assigned to a field of the opposite parity (See FIG.18).

In other words, when the current macroblock belongs to the top field,the value of the reference index for frame coding is assigned to the topfield out of the above two fields, as a reference index for fieldcoding. When the current macroblock belongs to the bottom field, thevalue of the reference index for frame coding is assigned to the bottomfield out of the above two fields, as a reference index for fieldcoding.

On the other hand, the decoding apparatus decodes the maximum number ofreference indices for frame coding and the assignment commands includedin the transmitted bit stream, and using them, it assigns the referencepictures and the reference indices in exactly the same manner as thecoding apparatus.

Structure of Coding Apparatus

FIG. 17 is a block diagram showing the structure of the coding apparatusin the fifth embodiment of the present invention. The coding apparatusin this figure is different from that in FIG. 1, in order to adapt tothe above (5.1) and (5.2), in that the former includes a referenceindex/picture number conversion unit 109 d, instead of the referenceindex/picture number conversion unit 109.

Example of Reference Index Assignment

FIG. 18 is an illustration showing correspondences of the first andsecond reference indices for field coding, updated from the first andsecond reference indices for frame coding as shown in FIG. 2, accordingto the above (5.1) and (5.2). As shown in FIG. 18, the value of thereference index for frame coding is applied to a field of the sameparity as a current macroblock as the reference index for field coding,while no index is applied to a field of the opposite parity.

Processing of Assigning Reference Indices

FIG. 19 is a flowchart showing the processing of assigning referenceindices executed by the reference index/picture number conversion unitin the coding apparatus. FIG. 19 is different from FIG. 6 in that S81 isadded instead of S27˜S30 and S82 is added instead of S31˜S34.

Structure of Decoding Apparatus

FIG. 20 is a block diagram showing the structure of the decodingapparatus in the fifth embodiment of the present invention. The decodingapparatus in FIG. 20 is different from that in FIG. 7 in that the formerincludes a reference index/picture number conversion unit 206 d, insteadof the reference index/picture number conversion unit 206.

According to the same operation as the mapping of (5.2), the referenceindex/picture number conversion unit 206 b executes a mapping of indicesfor field coding for top fields only if a current macroblock to bedecoded is in a top field and for bottom fields only if a currentmacroblock is in a bottom field, respectively.

Sixth Embodiment

Overview of Coding Apparatus and Decoding Apparatus

First, the overview of the coding apparatus and the decoding apparatusin the present embodiment will be explained.

The coding apparatus and the decoding apparatus in the presentembodiment perform MBAFF, and for that purpose, they handle the maximumnumber of the reference indices and the command sequence in thefollowing manners (6.1) and (6.2). Here, the reference indices and thecommands are same as those as shown in FIG. 37, and the maximum numberof the reference indices is same as that as shown in FIG. 39.

(6.1) As for the maximum number of reference indices, when both fieldcoding and frame coding are mixed, the coding apparatus describes notonly the maximum number of reference indices for frame coding but alsothe maximum number of reference indices for top field coding and themaximum number of reference indices for bottom field coding,respectively, in a bit stream to be transmitted.

The decoding apparatus uses the maximum number of reference indices fortop field coding and the maximum number of reference indices for bottomfield coding described in the bit stream.

(6.2) As for the command sequence, since it is same as that in (1.2),the explanation thereof is omitted. However, the reference indices fortop field coding are handled so as not to exceed the maximum numberdescribed in the bit stream. The same applies to the reference indicesfor bottom field coding.

On the other hand, the decoding apparatus decodes the maximum numbers ofthe reference indices for frame coding, top field coding and bottomfield coding and the assignment commands, which are included in thetransmitted bit stream, and using them, it assigns the referencepictures and the reference indices in exactly the same manner as thecoding apparatus.

Structure of Coding Apparatus and Decoding Apparatus

The coding apparatus and the decoding apparatus in the presentembodiment may be same as the coding apparatus and the decodingapparatus in the first embodiment. However, as the maximum number ofreference indices for top field coding and the maximum number ofreference indices for bottom field coding, they use the values describedin the bit stream, not the values obtained by doubling the values ofreference indices for frame coding.

Data Structure

FIG. 21 is a diagram showing the data structure of the bit stream in thesixth embodiment of the present invention. In this figure, the firstreference picture ref1 corresponds to Max_idx1 included in the picturecommon information, and the maximum number of reference indices forframe coding (Max_idx_frm), the maximum number of reference indices fortop field coding (Max_idx_top) and the maximum number of referenceindices for bottom field coding (Max_idx_btm) are described in Max_idx1.

FIG. 22 is an illustration showing an example of assigning the first andsecond reference indices to picture numbers of fields in a case of fieldcoding. In this figure, “5” is described in Max_idx_top, while “6” isdescribed in Max_idx_btm. In this way, the coding apparatus and thedecoding apparatus in the present embodiment can set the maximum numberof reference fields flexibly for top fields and bottom fields.

Note that the maximum number of reference indices for top field codingand the maximum number of reference indices for bottom field coding aredescribed in a bit stream separately (See (6.1)), but one maximum numbercommon to both top and bottom field coding may be described instead.

In (6.2), as in the case with (1.2), the value obtained by doubling thevalue of the reference index for frame coding is assigned to a field ofthe same parity as a current microblock to be coded, out of two fieldsthat make up one reference frame specified by the reference index andthe command for the frame, whereas the value obtained by doubling thevalue of that reference index for frame coding and adding 1 (x2+1) isassigned to another field of the opposite parity to the currentmicroblock, respectively, as reference indices for field coding (SeeFIG. 4). Instead, as in the case with (2.2), the value obtained bydoubling the value of the reference index for frame coding may beassigned to a top field, out of two fields that make up one referenceframe specified by the reference index and the command for the frame,and the value obtained by doubling the value of that reference index forframe coding and adding 1 (x2+1) may be assigned to a bottom field,respectively, as reference indices for field coding (See FIG. 9).

Seventh Embodiment

Overview of Coding Apparatus and Decoding Apparatus

First, the overview of the coding apparatus and the decoding apparatusin the present embodiment will be explained.

The coding apparatus and the decoding apparatus in the presentembodiment perform MBAFF, and for that purpose, they handle the maximumnumber of the reference indices and the command sequence in thefollowing manners (7.1) and (7.2). Here, the reference indices and thecommands are same as those as shown in FIG. 37, and the maximum numberof the reference indices is same as that as shown in FIG. 39.

(7.1) As for the maximum number of reference indices, since it ishandled in exactly the same manner as (6.1), the explanation thereof isomitted.

(7.2) As for the command sequence, the coding apparatus describes notonly the reference indices and the commands for frame coding but alsothe reference indices and the commands for top field coding and thereference indices and the commands for bottom field coding in a bitstream to be transmitted. The coding apparatus assigns the referenceindices for frame coding for the purpose of frame coding, while itassigns the reference indices for top field coding and the referenceindices for bottom field coding for the purpose of field coding.

On the other hand, the decoding apparatus decodes the maximum number ofreference indices and the assignment commands for frame coding, topfield coding and bottom field coding, included in the transmitted bitstream, and using them, it assigns the reference pictures and thereference indices in exactly the same manner as the coding apparatus.

Structure of Coding Apparatus

FIG. 23 is a block diagram showing the structure of the coding apparatusin the seventh embodiment of the present invention. The coding apparatusin this figure is different from that in FIG. 1 in that the formerincludes a reference index/picture number conversion unit 109 e, insteadof the reference index/picture number conversion unit 109.

FIG. 24 is a diagram showing an example of a data structure of a bitstream in the present embodiment. In this figure, idx_cmd1 is a set ofcommands for the first reference picture ref1, and includes idx_cmd_frm,idx_cmd_top and idx_cmd_btm. idx_cmd_frm is a command sequence forreference indices for frame coding. idx_cmd_top is a command sequencefor reference indices for top field coding. idx_cmd_btm is a commandsequence for reference indices for bottom field coding.

FIG. 25 is an illustration showing an example of assigning the first andsecond indices to picture numbers of fields in a case of field coding.In this figure, the reference indices for top field coding and thereference indices for bottom field coding can be independently assignedto arbitrary fields.

FIG. 26 is a diagram showing an example of correspondences betweenreference indices, commands and picture numbers of fields in a case ofFIG. 25.

FIG. 27 is a flowchart showing the processing of assigning referenceindices and commands executed by the reference index/picture numberconversion unit 109 e. As shown in this figure, the referenceindex/picture number conversion unit 109 e assigns reference indices andcommands for frame coding (S11), and when frame coding and field codingare mixed (S12), it assigns reference indices and commands for top fieldcoding (S93) and further assigns reference indices and commands forbottom field coding (S94).

Note that in FIG. 27, no command is assigned in S11, S93 and S94 whendefault reference indices are used.

Structure of Decoding Apparatus

FIG. 28 is a block diagram showing the structure of the decodingapparatus in the seventh embodiment of the present invention. FIG. 28includes a reference index/picture number conversion unit 206 e insteadof the reference index/picture number conversion unit 206 in FIG. 7. Thereference index/picture number conversion unit 206 e establishescorrespondences between picture numbers and reference indices for framecoding, top field coding and bottom field coding, respectively, usingindex assignment commands for them inputted from the bit stream analysisunit 201.

In the present embodiment, command sequences for top field coding andbottom field coding are described separately in a bit stream, but theymay be one common command sequence. FIG. 29 is a diagram showing thedata structure of the bit stream in that case. In this figure, idx_fldis a command sequence common to top field coding and bottom fieldcoding.

Note that the maximum number of reference indices for field coding asdescribed in (7.1) do not have to be specific to top field coding orbottom field coding, but may be common to top field coding and bottomfield coding.

Also, the reference indices and commands for field coding as describedin (7.2) do not have to be specific to top field coding or bottom fieldcoding, and may be common to top field coding and bottom field coding.

Also, the decoding apparatus in each of the above embodiments may createa reference table between reference indices for field coding and picturenumbers of fields before starting decoding of a slice, and refer to thetable when decoding a field-coded macroblock.

Eighth Embodiment

If a program for realizing the structures of the picture coding methodor the picture decoding method as shown in each of the above embodimentsis recorded on a memory medium such as a flexible disk, it becomespossible to perform the processing as shown in each of the embodimentseasily in an independent computer system.

FIGS. 30A, 30B and 30C are illustrations showing the case where thepresent invention is implemented in a computer system using a flexibledisk which stores the picture coding method or the picture decodingmethod of the above first to seventh embodiments.

FIG. 30B shows a front view and a cross-sectional view of an appearanceof a flexible disk, and the flexible disk itself, and FIG. 30A shows anexample of a physical format of a flexible disk as a recording mediumbody. The flexible disk FD is contained in a case F, and a plurality oftracks Tr are formed concentrically on the surface of the disk in theradius direction from the periphery and each track is divided into 16sectors Se in the angular direction. Therefore, as for the flexible diskstoring the above-mentioned program, the picture coding method as theprogram is recorded in an area allocated for it on the flexible disk FD.

FIG. 30C shows the structure for recording and reproducing the programon and from the flexible disk FD. When the program is recorded on theflexible disk FD, the picture coding method or the picture decodingmethod as a program is written in the flexible disk from the computersystem Cs via a flexible disk drive. When the picture coding method isconstructed in the computer system by the program on the flexible disk,the program is read out from the flexible disk using the flexible diskdrive and transferred to the computer system.

The above explanation is made on the assumption that a recording mediumis a flexible disk, but the same processing can also be performed usingan optical disk. In addition, the recording medium is not limited to aflexible disk and an optical disk, but any other medium such as an ICcard and a ROM cassette capable of recording a program can be used.

Ninth Embodiment

FIG. 31 to FIG. 34 are illustrations of devices for performing thecoding processing or the decoding processing as described in the aboveembodiments and a system using them.

FIG. 31 is a block diagram showing the overall configuration of acontent supply system ex100 for realizing content distribution service.The area for providing communication service is divided into cells ofdesired size, and base stations ex107 to ex110 which are fixed wirelessstations are placed in respective cells.

In this content supply system ex100, devices such as a computer ex111, aPDA (personal digital assistant) ex112, a camera ex113, a mobile phoneex114 and a camera-equipped mobile phone ex115 are connected to theInternet ex 101 via an Internet service provider ex102, a telephonenetwork ex104 and base stations ex107 to ex110.

However, the content supply system ex100 is not limited to theconfiguration as shown in FIG. 31, and a combination of any of them maybe connected. Also, each device may be connected directly to thetelephone network ex104, not through the base stations ex107 to ex110.

The camera ex113 is a device such as a digital video camera capable ofshooting moving pictures. The mobile phone may be a mobile phone of aPDC (Personal Digital Communications) system, a CDMA (Code DivisionMultiple Access) system, a W-CDMA (Wideband-Code Division MultipleAccess) system or a GSM (Global System for Mobile Communications)system, a PHS (Personal Handyphone system) or the like.

A streaming server ex103 is connected to the camera ex113 via the basestation ex109 and the telephone network ex104, which allows livedistribution or the like using the camera ex113 based on the coded datatransmitted from a user. Either the camera ex113 or the server fortransmitting the data may code the shot data. Also, the moving picturedata shot by a camera ex116 may be transmitted to the streaming serverex103 via the computer ex111. The camera ex116 is a device such as adigital camera capable of shooting still and moving pictures. Either thecamera ex116 or the computer ex111 may code the moving picture data. AnLSI ex117 included in the computer ex111 or the camera ex116 actuallyperforms coding processing. Software for coding and decoding movingpictures may be integrated into any type of storage medium (such as aCD-ROM, a flexible disk and a hard disk) that is a recording mediumwhich is readable by the computer ex111 or the like. Furthermore, thecamera-equipped mobile phone ex115 may transmit the moving picture data.This moving picture data is the data coded by the LSI included in themobile phone ex115.

The content supply system ex100 codes contents (such as a live musicvideo) shot by users using the camera ex113, the camera ex116 or thelike in the same manner as the above embodiment and transmits them tothe streaming server ex103, while the streaming server ex103 makesstream distribution of the content data to the clients at their request.The clients include the computer ex111, the PDA ex112, the camera ex113,the mobile phone ex114 and so on capable of decoding the above-mentionedcoded data. In the content supply system ex100, the clients can thusreceive and reproduce the coded data, and further the clients canreceive, decode and reproduce the data in real time so as to realizepersonal broadcasting.

When each device in this system performs coding or decoding, the movingpicture coding apparatus or the moving picture decoding apparatus, asshown in each of the above-mentioned embodiments, can be used.

A mobile phone will be explained as an example of the device.

FIG. 32 is a diagram showing the mobile phone ex115 that uses the movingpicture coding method and the moving picture decoding method explainedin the above embodiments. The mobile phone ex115 has an antenna ex201for sending and receiving radio waves to and from the base stationex110, a camera unit ex203 such as a CCD camera capable of shootingvideo and still pictures, a display unit ex202 such as a liquid crystaldisplay for displaying the data obtained by decoding video and the likeshot by the camera unit ex203 and received via the antenna ex201, a bodyunit including a set of operation keys ex204, a voice output unit ex208such as a speaker for outputting voices, a voice input unit 205 such asa microphone for inputting voices, a storage medium ex207 for storingcoded or decoded data such as data of moving or still pictures shot bythe camera, and text data and data of moving or still pictures ofreceived e-mails, and a slot unit ex206 for attaching the storage mediumex207 to the mobile phone ex115. The storage medium ex207 includes aflash memory element, a kind of EEPROM (Electrically Erasable andProgrammable Read Only Memory) that is an electrically erasable andrewritable nonvolatile memory, in a plastic case such as an SD card.

The mobile phone ex115 will be further explained with reference to FIG.33. In the mobile phone ex115, a main control unit ex311 for overallcontrolling the display unit ex202 and the body unit including operationkeys ex204 is connected to a power supply circuit unit ex310, anoperation input control unit ex304, a picture coding unit ex312, acamera interface unit ex303, an LCD (Liquid Crystal Display) controlunit ex302, a picture decoding unit ex309, a multiplex/demultiplex unitex308, a record/reproduce unit ex307, a modem circuit unit ex306 and avoice processing unit ex305, and they are connected to each other via asynchronous bus ex313.

When a call-end key or a power key is turned ON by a user's operation,the power supply circuit unit ex310 supplies respective units with powerfrom a battery pack so as to activate the camera-equipped digital mobilephone ex115 for making it into a ready state.

In the mobile phone ex115, the voice processing unit ex305 converts thevoice signals received by the voice input unit ex205 in conversationmode into digital voice data under the control of the main control unitex311 including a CPU, ROM and RAM, the modem circuit unit ex306performs spread spectrum processing of the digital voice data, and thesend/receive circuit unit ex301 performs digital-to-analog conversionand frequency transform of the data, so as to transmit it via theantenna ex201. Also, in the mobile phone ex115, after the data receivedby the antenna ex201 in conversation mode is amplified and performed offrequency transform and analog-to-digital conversion, the modem circuitunit ex306 performs inverse spread spectrum processing of the data, andthe voice processing unit ex305 converts it into analog voice data, soas to output it via the voice output unit 208.

Furthermore, when transmitting e-mail in data communication mode, thetext data of the e-mail inputted by operating the operation keys ex204on the body unit is sent out to the main control unit ex311 via theoperation input control unit ex304. In the main control unit ex311,after the modem circuit unit ex306 performs spread spectrum processingof the text data and the send/receive circuit unit ex301 performsdigital-to-analog conversion and frequency transform for it, the data istransmitted to the base station ex110 via the antenna ex201.

When picture data is transmitted in data communication mode, the picturedata shot by the camera unit ex203 is supplied to the picture codingunit ex312 via the camera interface unit ex303. When the picture data isnot transmitted, it is also possible to display the picture data shot bythe camera unit ex203 directly on the display unit 202 via the camerainterface unit ex303 and the LCD control unit ex302.

The picture coding unit ex312, which includes the picture codingapparatus as explained in the present invention, compresses and codesthe picture data supplied from the camera unit ex203 by the codingmethod used for the picture coding apparatus as shown in the aboveembodiments so as to transform it into coded picture data, and sends itout to the multiplex/demultiplex unit ex308. At this time, the mobilephone ex115 sends out the voices received by the voice input unit ex205during shooting by the camera unit ex203 to the multiplex/demultiplexunit ex308 as digital voice data via the voice processing unit ex305.

The multiplex/demultiplex unit ex308 multiplexes the coded picture datasupplied from the picture coding unit ex312 and the voice data suppliedfrom the voice processing unit ex305 by a predetermined method, themodem circuit unit ex306 performs spread spectrum processing of themultiplexed data obtained as a result of the multiplexing, and thesend/receive circuit unit ex301 performs digital-to-analog conversionand frequency transform of the data for transmitting via the antennaex201.

As for receiving data of a moving picture file which is linked to a Webpage or the like in data communication mode, the modem circuit unitex306 performs inverse spread spectrum processing of the signal receivedfrom the base station ex110 via the antenna ex201, and sends out themultiplexed data obtained as a result of the processing to themultiplex/demultiplex unit ex308.

In order to decode the multiplexed data received via the antenna ex201,the multiplex/demultiplex unit ex308 separates the multiplexed data intoa bit stream of picture data and a bit stream of voice data, andsupplies the coded picture data to the picture decoding unit ex309 andthe voice data to the voice processing unit ex305 respectively via thesynchronous bus ex313.

Next, the picture decoding unit ex309, which includes the picturedecoding apparatus as explained in the present invention, decodes thebit stream of picture data by the decoding method corresponding to thecoding method as shown in the above-mentioned embodiments to generatereproduced moving picture data, and supplies this data to the displayunit ex202 via the LCD control unit ex302, and thus moving picture dataincluded in a moving picture file linked to a Web page, for instance, isdisplayed. At the same time, the voice processing unit ex305 convertsthe voice data into analog voice data, and supplies this data to thevoice output unit ex208, and thus voice data included in a movingpicture file linked to a Web page, for instance, is reproduced.

The present invention is not limited to the above-mentioned system, andat least either the picture coding apparatus or the picture decodingapparatus in the above-mentioned embodiments can be incorporated into asystem for digital broadcasting as shown in FIG. 34. Such ground-basedor satellite digital broadcasting has been in the news lately. Morespecifically, a coded bit stream of video information is transmittedfrom a broadcast station ex409 to a communication or broadcast satelliteex410 via radio waves. Upon receipt of it, the broadcast satellite ex410transmits radio waves for broadcasting, a home-use antenna ex406 with asatellite broadcast reception function receives the radio waves, and atelevision (receiver) ex401 or a set top box (STB) ex407 decodes the bitstream for reproduction. The picture decoding apparatus as shown in theabove-mentioned embodiments can be implemented in the reproductionapparatus ex403 for reading off and decoding the bit stream recorded ona storage medium ex402 that is a recording medium such as a CD and DVD.In this case, the reproduced video signals are displayed on a monitorex404. It is also conceived to implement the picture decoding apparatusin the set top box ex407 connected to a cable ex405 for a cabletelevision or the antenna ex406 for satellite and/or ground-basedbroadcasting so as to reproduce them on a monitor ex408 of thetelevision ex401. The picture decoding apparatus may be incorporatedinto the television, not in the set top box. Or, a car ex412 having anantenna ex411 can receive signals from the satellite ex410, the basestation ex107 or the like for reproducing moving pictures on a displaydevice such as a car navigation system ex413 in the car ex412.

Furthermore, the picture coding apparatus as shown in theabove-mentioned embodiments can code picture signals for recording on arecording medium. As a concrete example, there is a recorder ex420 suchas a DVD recorder for recording picture signals on a DVD disc ex421 anda disk recorder for recording them on a hard disk. They can be recordedon an SD card ex422. If the recorder ex420 includes the picture decodingapparatus as shown in the above-mentioned embodiments, the picturesignals recorded on the DVD disc ex421 or the SD card ex422 can bereproduced for display on the monitor ex408.

As the structure of the car navigation system ex413, the structurewithout the camera unit ex203, the camera interface unit ex303 and thepicture coding unit ex312, out of the units shown in FIG. 33, isconceivable. The same applies to the computer ex111, the television(receiver) ex401 and others.

In addition, three types of implementations can be conceived for aterminal such as the above-mentioned mobile phone ex114; asending/receiving terminal including both an encoder and a decoder, asending terminal including an encoder only, and a receiving terminalincluding a decoder only.

As described above, it is possible to use the moving picture codingmethod or the moving picture decoding method in the above-mentionedembodiments in any of the above-mentioned apparatuses and systems, andusing this method, the effects described in the above embodiments can beobtained.

It should be noted that the present invention is not limited to theabove embodiments, and many variations or modifications thereof arepossible without departing from the scope of the invention.

INDUSTRIAL APPLICABILITY

The present invention is suitable for a picture coding apparatus forperforming coding with switching between frame coding and field codingon a block-by-block basis in a picture and a picture decoding apparatus.More specifically, it is suitable for a Web server for distributingmoving pictures, a network terminal for receiving them, a digital camerafor recording and replaying moving pictures, a camera-equipped mobilephone, a DVD recorder/player, a PDA, a personal computer and the like.

The invention claimed is:
 1. A decoding apparatus which decodes an inputbit stream of a coded B frame by adaptively switching, on ablock-by-block basis, between frame decoding and field decoding, whereinthe input bit stream of the coded B frame is generated by: 1) generatingfirst and second maximum number information indicating a first maximumnumber of first frame indices and a second maximum number of secondframe indices, the first maximum number indicating a maximum number ofthe first frame index for frame coding and the second maximum numberindicating a maximum number of the second frame index for frame coding;2) generating first commands and second commands, the first commandsindicating correspondence between the first frame indices and firstreference frames, each of the first frame indices being an index fordesignating the first reference frame for a current block to be codedand the second commands indicating correspondence between the secondframe indices and second reference frames, each of the second frameindices being an index for designating the second reference frame forthe current block to be coded; 3) (i) generating a first reference indexcorresponding to the first reference frame for the current block to becoded and a second reference index corresponding to the second referenceframe for the current block to be coded when frame coding is selectedfor the current block to be coded, the first and second reference framesbeing referred to when the current block is coded through motioncompensation using frame coding, and (ii) generating a first referenceindex corresponding to the first reference field for the current blockto be coded and a second reference index corresponding to the secondreference field for the current block to be coded when field coding isselected for the current block to be coded, the first and secondreference fields being referred to when the current block is codedthrough motion compensation using field coding; and 4) coding aprediction error for the current block to be coded, said decodingapparatus comprising: a maximum number obtaining unit operable toobtain, from a bit stream, a first maximum number of first frame indicesand a second maximum number of second frame indices, the first maximumnumber indicating a maximum number of the first frame index for framedecoding and the second maximum number indicating a maximum number ofthe second frame index for frame decoding; a command obtaining unitoperable to obtain, from the bit stream, first commands and secondcommands, the first commands indicating correspondence between the firstframe indices and first reference frames, each of the first frameindices being an index for designating the first reference frame for acurrent block to be decoded and the second commands indicatingcorrespondence between the second frame indices and second referenceframes, each of the second frame indices being an index for designatingthe second reference frame for the current block to be decoded; adetermining unit operable to determine (i) a maximum number of firstfield indices for field decoding to be double a value of the firstmaximum number of the first frame indices and operable to determine (ii)a maximum number of second field indices for field decoding to be doublea value of the second maximum number of the second frame indices; aswitching unit operable to adaptively switch, on a block-by-block basis,between frame decoding and field decoding; an extracting unit operableto extract, from the bit stream, a first reference index and a secondreference index for the current block to be decoded; a referenceframe/field specifying unit operable to (i) specify a first referenceframe corresponding to the extracted first reference index and a secondreference frame corresponding to the extracted second reference indexwhen frame decoding is performed for the current block to be decoded,the first and second reference frames being referred to when the currentblock is decoded through motion compensation using frame decoding, (ii)specify a first reference field corresponding to the extracted firstreference index for a top field of the current block to be decoded and asecond reference field corresponding to the extracted second referenceindex for the top field of the current block to be decoded when fielddecoding is performed for the top field of the current block to bedecoded, the first and second reference fields being referred to whenthe top field of the current block is decoded through motioncompensation using field decoding, and (iii) specify a first referencefield corresponding to the extracted first reference index for a bottomfield of the current block to be decoded and a second reference fieldcorresponding to the extracted second reference index for the bottomfield of the current block to be decoded when field decoding isperformed for the bottom field of the current block to be decoded, thefirst and second reference fields being referred to when the bottomfield of the current block is decoded through motion compensation usingfield decoding; and a decoding unit operable to decode a predictionerror for the current block to be decoded to obtain a recovered currentblock, wherein said reference frame/field specifying unit for the topfield of the current block to be decoded is further operable to: (i)extract the first reference index for field decoding so that a value ofthe first reference index is not greater than a maximum value of thefirst field indices for field decoding, and extract the second referenceindex for field decoding so that a value of the second reference indexis not greater than a maximum value of the second field indices forfield decoding; (ii) specify, as the first reference field for fielddecoding, a top field out of two fields that make up the first referenceframe specified according to the first frame index, in the case where avalue of the extracted reference index is double a value of the firstframe index, and specify, as the first reference field for fielddecoding, a bottom field out of the two fields that make up the firstreference frame specified according to the first frame index, in thecase where the value of the extracted reference index is double thevalue of the first frame index, plus one; and (iii) specify, as thesecond reference field for field decoding, a top field out of two fieldsthat make up the second reference frame specified according to thesecond frame index, in the case where a value of the extracted referenceindex is double a value of the second frame index, and specify, as thesecond reference field for field decoding, a bottom field out of the twofields that make up the second reference frame specified according tothe second frame index, in the case where the value of the extractedreference index is double the value of the second frame index, plus one,and wherein said reference frame/field specifying unit for the bottomfield of the current block to be decoded is further operable to: (i)extract the first reference index for field decoding so that a value ofthe first reference index is not greater than a maximum value of thefirst field indices for field decoding, and extract the second referenceindex for field decoding so that a value of the second reference indexis not greater than a maximum value of the second field indices forfield decoding; (ii) specify, as the first reference field for fielddecoding, a bottom field out of two fields that make up the firstreference frame specified according to the first frame index, in thecase where a value of the extracted reference index is double a value ofthe first frame index, and specify, as the first reference field forfield decoding, a top field out of the two fields that make up the firstreference frame specified according to the first frame index, in thecase where the value of the extracted reference index is double thevalue of the first frame index, plus one; and (iii) specify, as thesecond reference field for field decoding, a bottom field out of twofields that make up the second reference frame specified according tothe second frame index, in the case where a value of the extractedreference index is double a value of the second frame index, andspecify, as the second reference field for field decoding, a top fieldout of the two fields that make up the second reference frame specifiedaccording to the second frame index, in the case where the value of theextracted reference index is double the value of the second frame index,plus one.